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METHOD AND APPARATUS 
FOR NOISE REDUCTION USING CAPTURED IMAGES 

Background Of The Invention 

5 1. Field of the Invention 

The present invention relates generally to reducing phase noise and distortion in 
the conversion of an analog video signal to a digital video signal. In particular, the 

0 present invention is directed to technology that reduces phase noise and distortion in 

01 the digitally converted image by selectively capturing reference frames of the image in 
t§ accordance with a change threshold. 

01 

I'* 2. Background Information 

J;^ Presentations using multimedia projection display systems have become popular 

f : for purposes such as sales demonstrations, business meetings, and classroom 
II sessions. In a common mode of operation, multimedia projection display systems 
receive analog video signals from a multimedia device such as a personal computer 
(PC). The video signals represent still, partial-, or full-motion display images of the type 
rendered by the PC. The analog video signals are converted into digital video signals to 
control a digitally-driven display object, such as a transmissive liquid crystal display 
20 (LCD) or digital mirror device (DMD), to form the display images for projection onto a 
display screen. 

Two common types of multimedia projection display systems are LCD projectors 
and LCD projection panels. An LCD projector includes a transmissive LCD, a light 
source, and projection optics to form and project display images in the manner 
25 described above. An LCD projection panel includes a similar transmissive LCD to form 
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the display image, but operates with a conventional overhead projector (OHP) having a 
light source and projection optics, to project the display image onto a display screen. 
Examples of such LCD projectors and LCD projection panels are sold under the 
respective trademarks LITEPRO and PANELBOOK by In Focus Systems, Inc. of 

5 Wilsonville, Oreg., the assignee of the present application. 

Because the LCDs and DMDs used in multimedia projection display systems 
require digital video signals, the multimedia projection display system normally has an 
analog to digital (A/D) signal converter for converting the PC-generated analog video 

D signals into a digital format suitable for driving the LCD, DMD or other type of display 

tB object. 

y The PC-generated analog video signal is actually comprised of three analog 

|V image data signals, one for each of the primary colors, red, green and blue, and a digital 

I " timing signal, which may include a pulsed horizontal synchronizing signal (H sync ) as well 

as a pulsed vertical synchronizing signal (V sync ), or a composite sync signal. The 

individual analog image data signals representing the colors are generated from bit data 

O in a memory portion of the PC, using three digital-to-analog (D/A) converters, one for 

each of the colors, red, green, and blue. 

A complete image is typically displayed during a time interval known as a "video 

frame period." With reference to Fig. 4, each video frame 9 is usually produced to have 

20 a central active video region 11 surrounded by an inactive ("blanked") margin 13. The 

central active video region 1 1 is composed of a number of horizontal and vertical picture 

elements (pixels), typically ranging from 640 X 480 pixels to 1600 X 1200 pixels, 

referred to as the display resolution mode. As the image changes the pixels are 

activated on and off, with the central active video region 1 1 being refreshed at rates of 

25 60 to 100 Hz, as dictated by the constraints of the display object (e.g. the LCD, DMD, 

etc.) used in the multimedia projection display system. Since compatibility with the 
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various PC display resolution modes is a desirable feature for multimedia projection 
display systems, the A/D signal converter typically includes an interface that is capable 
of converting analog video signals having a variety of resolution modes. 

The A/D signal converter is usually combined with a phase-locked loop (PLL), 
5 which may comprise a phase comparator, a low-pass loop filter, and a voltage- 
controlled oscillator (VCO) formed in a loop to generate a feedback signal that locks into 
the H sync pulsed horizontal synchronizing signal. In order to generate a selected 
multiple n of pixel clock pulses for each period of H synCi a divide-by-n counter is added to 
the feedback loop between the VCO output and the phase comparator, 
f 0 An example of a pixel clock waveform 4 is shown in FIG. 1 . The number n of 

^ individual pixel clock pulses per H sync pulse may be set by reference to the resolution 
mode of the analog video source. To set the resolution mode, certain characteristics of 
the analog video signal, such as H sync and V synC} may be used to refer to a resolution 
B mode look-up table stored in the projection display system's CPU. The number n should 
II be set to equal the number of pixel data components in each horizontal line of the 
O scanned analog signal, including those in the central active video region 1 1 and the 
blanked margin regions 1 3 (see FIG. 4) on either side of the central active video region 
11. For example, for a screen resolution of 640 X 480, the number n may be set at 
about 800 to include the blanked regions 13 on either side of the 640 pixel-wide central 
20 active video region 1 1 . Thus, the pixel clock would sample the continuous stream of 
analog image data 800 times along each horizontal line of the frame. 

FIG. 1 further shows the desired relationship between the analog video data 
signal 1 and the pixel clock signal 4. The number n of pixel clocks 5 is set to establish a 
one-to-one relationship between pixel clock pulses 5 and pixel data components 2 of 
25 the analog data signal 1 . This one-to-one relationship requires that the pixel clock signal 

frequency be equal to the analog video data signal frequency. Under this relationship, 
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each pixel data component 2 of the analog signal is sampled by a single pixel clock 
pulse 5, which reads the instantaneous voltage value of the pixel data component so 
that it can be digitized. The stream of digitized instantaneous voltage values form the 
digital video data signal, which is addressed to the LCD, DMD, or other type of display 
5 object, to appropriately set pixels to blank (black) or to selected activated (non-black) 
status to replicate the image defined by the analog video data signal. 

One problem with the above-described A/D conversion is that it is often imperfect 
due to errors in the pixel clock sampling of the analog signal. With reference to FIG. 1, 
n since the pixel clock pulses 5 have "jitter" zones 6 at their leading and trailing edges, the 
|3 clock pulses 5 should be registered with the centers of the pixel data components 2, so 
f j j that the sampling is not randomly pushed by the jitter into the transition regions 3 of the 
J jl analog video data signal 1 . If the pixel clock pulses 5 are not properly registered, the 
W resulting sampling imprecision gives rise to frequency (also know as "tracking") and 
O "phase" errors, both of which introduce noise that may degrade the quality of the LCD, 
IS DMD, or other display object's image. 

p FIG. 2 illustrates a tracking error that results from the improperly setting of the 

number n of pixel clocks. As discussed above, the number n of pixel clocks should be 
equal to the number of pixel data components 2 of each horizontal line of analog video 
data signal 1 . As illustrated, the improper setting of n results in the pixel data 

20 components 2 not being sampled at a consistent point. Specifically, n is set too large in 
clock signal 4' (i.e. the clock signal frequency is too high). The resulting crowding of the 
pixel clock pulses 5' yields an additive leftward drift of the pixel clock pulses 5' relative to 
the pixel data components 2 of the analog video data signal 1 . Such drift causes 
sampling in the transition regions 3. For instance, as indicated by positional bracket A, 

25 the leading edges 7' of the third through the sixth clock pulses 5' sample in transition 

zones 3 of the analog video data signal 1 . Accordingly, the transition zone data will be 
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erroneous and the image information from adjacent non-sampled pixel data 
components 2 will be missing from the digitized video signal. If n is erroneously set 
large enough, the pixel clock pulses may be so crowded that individual analog pixel 
data components 2 may be double-sampled. On the other hand, if n is erroneously set 
5 too small (i.e. the pixel clock signal frequency is too low), the dispersion of the pixel 
clock pulses results in a rightward drift wherein sampling may also occur in the 
transition regions. In all of these cases, the erroneous sampling provides erroneous 
video signal data that generates unwanted distortion that degrades the LCD or other 

o 

!jj display object video image quality. 

f H FIG. 3 illustrates that phase error may occur even when the pixel clock signal 

p» frequency equals the analog video data signal frequency (i.e. even when there is no 

Q tracking error). For example, as shown in pixel clock signal 4" in FIG, 3, the clock 

jl phase may be erroneously set such that every pixel clock pulse samples a transition 

region 3 of the analog video data signal. Leading edge jitter makes such phase error 

+5 more likely, since if the jitter zones 6" straddle the intersections 8 of the pixel data 

O components 2 and transition regions 3 of the analog video data signal 1 , the voltage will 

be randomly sampled on either side of the intersection 8. In any case, phase error is 

undesirable since it generates unwanted phase noise, or "snow" in the LCD, DMD, or 

other display object video image. 

20 Prior art techniques for reducing noise have focused on providing feedback to the 

pixel clock in order to eliminate the above-described tracking and phase errors. For 

example, a current projection display system may include an image capture circuit that 

automatically eliminates phase and tracking errors by monitoring the actual active 

image width of the analog video data signal measured by the number n of pixel clocks, 

25 and adjusting the frequency and phase components of the pixel clock signal until the 

expected width of the image matches the actual width. A current projection display 

- 6- 

M & A FOR NOISE REDUCTION 
USING CAPTURED IMAGES 
EL034440050US DJC/mwb 



It 

Attorney Docket No. 004589.P003 

system may also include an image capture circuit that automatically eliminates phase 
and tracking errors by monitoring a selected pixel data component at the edge of the 
central active video region that is horizontally stationary from frame to frame of the 
analog video data, and automatically iteratively adjusting the pixel clock pulse until a 

5 pixel clock pulse is centrally registered with the selected pixel data component. Both of 
these prior art techniques used in current projection display systems are described in 
U.S. Pat. Nos. 5,767,916 and 5,805,233, assigned to In Focus Systems, Inc. of 
Wilsonville, Oreg., the assignee of the present invention. 

r =j The image capture circuit of a current projection display system includes a 

1 Jf programmable delay device, a PLL, a divide-by-n-counter, an A/D converter, and an 

y '= 

Ml ASIC (Application Specific Integrated Circuit) that contains the image edge or width 
pi detection circuitry that provides the feedback necessary to correct the pixel clock. 

In operation, the A/D converter samples (reads) the instantaneous voltage value 
p of the analog video data signal at the leading edge of each of the pixel clocks, thereby 
li generating a series of sampled data signal values. The AID converter then quantizes 
□ the sampled values by matching each value to one of a series of preselected voltage 
amplitude levels, which have corresponding numerical values. The numerical values 
are represented digitally and coded to establish 8-bit data for each of the colors, red, 
green, and blue. The three eight-bit color data signals are input through the three 
20 respective color data signal channels to the ASIC. A window random access memory 
(WRAM) is connected between the ASIC and the LCD, DMD, or other display object 
that ultimately receives the output from the A/D converter. At the display object, the 
coded color data signals set pixels to blank (black) or to specific activated (non-black) 
status corresponding to the sampled voltage level. A microcontroller, which is part of 
25 the current projection display system CPU, uses the feedback provided by the ASIC to 
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control the programmable delay device and divide-by-n-counter to change the pixel 
clock settings and eliminate the phase and tracking errors. 

Despite the above-described improvements in eliminating phase and tracking 
errors, current projection display systems are still not optimum. Particularly frustrating is 
5 the fact that the causes of the phase noise and distortion may dynamically change as 
the environment changes. Thus, any given technique for reducing or eliminating the 
distortion may or may not be effective from one moment to the next. 

Another problem is that a digitized video frame representing the complete image 
-h displayed during the video frame period must be stored within the WRAM until it is 
IS transmitted to the display object. Since the video frames are typically stored in WRAM 
JjiJ at a faster rate than they are transmitted to the display object, the WRAM must include 

enough frame buffer capacity or memory to store a number of video frames at once. 
!L For example, each video frame may be stored to the WRAM at the rate of 80 Hz (i.e. 80 
M frames per second), but may only be output to the display object at the rate of 60 Hz, as 
i§ dictated by the refresh rate of the particular display object (e.g. the LCD, DMD, etc.). 
O For even the lowest resolution video display systems, storing video frames in 

WRAM results in the storage of a significantly large amount of data. Since a video 
frame represents the complete image, it contains data representing every pixel in the 
central active video region. The optical state of each pixel in the central active video 
20 region, i.e. its color or shade of gray, is described by several bits of data; the exact 

number of bits depends upon the desired number of colors or shades of gray that are to 
be displayed. For example, a typical LCD may have 480 rows and 640 columns that 
intersect to form a matrix of 307,200 pixels corresponding to the pixels in the central 
active video region. Consequently, if 8 bits are required to specify each of the three 
25 colors comprising each pixel, then 921 ,600 bytes of image data (3 bytes X 307,200 
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pixels), or nearly 1 megabyte, are required to characterize a single digitized video 
frame. 

The problem of having to provide a necessarily large WRAM storage capacity is 
compounded by the fact that the WRAM is expensive, which adds significantly to the 
5 cost of the multimedia projection display system. Additionally, the WRAM takes up a 
large amount of board space. Moreover, the sheer volume of digitized video frames 
generated by the multimedia projection display system requires a significantly large 
transmission bandwidth between the WRAM and the LCD, DMD, or other display object 
O Large transmission bandwidth requirements present a problem when designing 
tS networked presentation projector applications. For example, in a wireless network 
y environment, including wireless networks designed in accordance with the IEE 802.1 1 
[1 Wireless LAN Standard, or the Draft Specification of Bluetooth; A Global Specification 
I " for Wireless Connectivity, promulgated by the Bluetooth Special Interest Group, or even 
in a conventional networked environment where the LCD, DMD, or other display object 
f§ is capable of providing its own frame memory, it is desirable to reduce the transmission 
y bandwidth requirements where possible to enable the development of networked 
presentation projector applications. 

Accordingly, it is desirable to provide a multimedia projection display system that 
not only reduces the noise and distortion in the digitally sampled image, but also 
20 reduces the transmission bandwidth requirements between the WRAM and the display 
object. 
SUMMARY 

According to one aspect of the invention, a method is provided in which a digitally 

converted image is displayed free of phase noise and distortion and using a reduced 

25 amount of stored frame memory by displaying only a selected subset of the video 

frames that represent the digitally converted image. The selected subset is comprised 
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of reference video frames, where a new reference frame is captured for display only 
after determining that a pixel of an inbound video frame exceeds a certain threshold of 
change as compared to a corresponding pixel of the previously captured reference 
frame. 

In accordance with other aspects of the present invention, apparatus are 
provided for carrying out the above and other methods. 

BRIEF DESCRIPTION OF DRAWINGS 

The present invention will be described by way of exemplary embodiments, but 
not limitations, illustrated in the accompanying drawings in which like references denote 
similar elements, and in which: 

FIG. 1 depicts an analog video data signal waveform and a pixel clock signal 
waveform in a desired relationship where no tracking or phase error would be result. 

FIG. 2 depicts an analog video data signal waveform and a pixel clock signal 
waveform in a relationship where tracking error would result. 

FIG. 3 depicts an analog video data signal waveform and a pixel clock signal 
waveform in a relationship where phase error would result. 

FIG. 4 depicts an exemplary video display, showing a central active video region 
surrounded by an inactive margin region. 

FIG. 5 is an overall schematic view of a multimedia source connected to a 
multimedia projection display system, further depicting a reference frame capture circuit 
in accordance with one embodiment of the present invention. 

FIG. 6 is a schematic view of a phase-locked-loop (PLL) circuit used in analog 
video signal capture in accordance with one embodiment of the present invention. 

FIG. 7 schematically depicts a pixel value calculator in accordance with one 
embodiment of the present invention. 
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FIG. 8 is an block diagram of the logical components of the reference frame 
capture circuit shown in FIG. 5, in accordance with one embodiment of the invention. 

FIG. 9 is a flow diagram showing a reference frame capture method used in the 
reference frame capture circuit shown in FIG. 5, in accordance with one embodiment of 
the present invention. 

FIG. 10 illustrates an example implementation in the form of a tabulation of a 
series of inbound frames and lines and the corresponding pixel value comparisons and 
resulting actions in accordance with one embodiment of the present invention. 

FIG. 11 illustrates a pixel value comparison of individual red, green, and blue 
pixel values in decimal notation in accordance with one embodiment of the present 
invention. 

DETAILED DESCRIPTION OF THE INVENTION 

In the following description various aspects of the present invention, a method and 
apparatus for reducing phase noise and distortion in a digitally converted image, will be 
described. Specific details will be set forth in order to provide a thorough understanding 
of the present invention. However, it will be apparent to those skilled in the art that the 
present invention may be practiced with only some or all of the described aspects of the 
present invention, and with or without some or all of the specific details. In some 
instances, well known features may be omitted or simplified in order not to obscure the 
present invention. 

Various operations will be described as multiple discrete steps performed in turn in 
a manner that is most helpful in understanding the present invention. However, the order 
of description should not be construed as to imply that these operations are necessarily 
performed in the order they are presented, or even order dependent. Lastly, repeated 
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usage of the phrase "in one embodiment" does not necessarily refer to the same 
embodiment, although it may. 

Devices that generate digitally converted images generally refresh the screen 
display at rates of 60 -80 Hz (60-80 frames per second) or more to provide a quality 
image and to avoid eye fatigue. However, the human eye perceives only a portion of 
the frames, approximately one every 16 milliseconds, or about 1/6 th of the frames 
displayed at 60 Hz. Therefore, it is possible to reduce the number of frames sent to the 
device without impairing the perceived quality of the image. A reduction in the number 
of frames sent to the device is particularly desirable in a wireless network environment, 
or where the device is capable of providing its own frame memory, as it reduces the 
amount of bandwidth necessary to transmit the frame data between the multimedia 
projection display system and the display object. 

The method of the present invention is based in part on the observation that 
selectively sending new frames only when there is a perceptively significant change in 
the frame data not only reduces the number of frames sent to the device, thereby 
reducing bandwidth requirements, but also improves the overall quality of the perceived 
image by eliminating unnecessary phase noise and distortion in the digitally converted 
image. 

Referring now to FIG. 5, wherein an overall schematic view of a multimedia 

source connected to a multimedia projection display system depicting a reference frame 

capture circuit in accordance with one embodiment of the present invention is 

illustrated. As shown, the projection display system 10 includes an image capture 

circuit 14 that automatically eliminates phase noise, tracking error, and image distortion. 

A microcontroller 16, which is part of a display system CPU 17, controls the image 

capture circuit 14. In the illustrated embodiment, the image capture circuit 14 includes a 

programmable delay device 18, a PLL 20, a divide-by-n-counter (counter) 22, an A/D 
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converter 24, and an ASIC (Application Specific Integrated Circuit) that contains an 
image edge detection circuit 26 and the reference frame capture circuit 27. The 
microcontroller 16 executes a firmware program that runs the ASIC circuits 26 and 27. 
Turning to the signal path to the display object, such as an active matrix LCD 28, 
5 shown in FIG. 5, the line advance conductor 53, the digital video data signal channels 
56a, 56b, 56c, and the frame advance conductor 58 are connected to an input port of a 
window random access memory (WRAM) 132. The WRAM has an output port 
connected to a display object control module 134, which in turn is connected to the 
^ LCD, DMD, or other display object 28. The connection between the WRAM output port 
ID and the display object control module 1 34 and LCD, DMD, or other display object 28, 
in may be a conventional wired connection or a wireless network connection including, for 
01 example, a wireless network connection designed in accordance with the I EE 802.1 1 
y Wireless LAN Standard or in accordance with the Draft Specification of Bluetooth: A 
P Global Specification for Wireless Connectivity, promulgated by the Bluetooth Special 
t8 Interest Group. The LCD is preferably a 6.4 inch active matrix LCD, such as model 
p LQ64P31 1 made by Sharp. 

- Based on the output of the edge detection circuit 26 the microcontroller 1 6 further 

controls the delay device 18 and the counter 22 to eliminate phase noise and tracking 
errors. Based on the output of the reference frame capture circuit 27, the 

20 microcontroller 16 further controls the capture to frame memory in WRAM 132 of a 
reference frame from the frames output from the A/D converter 24 so as to eliminate 
phase noise and image distortion. The number of reference frames captured depends 
on the type of image being processed and a threshold setting such that the rate of 
capture of a reference frame is lower when the image is static (e.g. fixed image 

25 presentations) , and higher when the image is dynamic (e.g. full motion video). The 

threshold setting is pre-selected to provide for the maximum elimination of phase noise 
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and distortion, while providing a sufficient number of reference frames for proper 
replication of the digitally converted image at the display object 28. It should be 
understood that the outputs of the edge detection circuit 26 and reference frame capture 
circuit 27 are both independent and complementary such that the reference frame 
capture circuit 27 may be used alone or in combination with the edge detection circuit 
26 or other methods to improve the quality of the digitally converted image displayed by 
the display object 28. 

As shown in FIG. 5, for purposes of video signal transmission, the multimedia 
source 12 is connected to the projection display system 10 through a video source 
cable 30 which is shown in exploded schematic form. The cable 30 is of conventional 
design and includes multiple distinct conductors that are shielded together, including 
three separate channels 32a, 32b, 32c for carrying analog signals corresponding to red, 
green, and blue (RGB) color video components, and two conductors 34, 36 carrying the 
H sy nc and V syn c signals, respectively. 

Turning to the details of the image capture circuit 14, the microcontroller 16 is 
connected to the delay device 18 by a bus 38, to the counter 22 by a bus 40, and to the 
ASIC 26 by a bus 42. A mode identification counter 43, which is connected to H syn c and 
Vsync through conductors 45 and 47, respectively, may be located in the microcontroller 
or the ASIC. The mode identification counter 43 may also be provided independent of 
the microcontroller. A preferred microcontroller 16 is model MC6833 1, made by 
Motorola. The delay device has an input connected to the H syn c conductor 34, and an 
output connected to the PLL through conductor 44. The preferred delay device is model 
No. DS1 0205-25, made by the Dallas Corporation. 

As shown in detail in FIG. 6, the PLL 20 is of conventional design and includes a 

phase comparator 46, a low-pass filter 48, and a VCO pixel clock signal generator 50. A 

feedback loop 52 provided with the counter 22 connects the VCO output 55 and the 
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phase comparator 46. The counter output 51 is connected to the ASIC 26 through a 
conductor 53, and the VCO output 55 is connected to the ASIC 26 and the A/D 
converter 24 through conductor 54. The preferred PLL is model ICS 1522 made by ICS. 
The counter is preferably a part of the ASIC 26. 

Referring again to FIG. 5, the three analog video data signal channels 32a, 32b, 
32c are connected to the A/D converter input. The A/D converter 24 includes three 
separate conventional A/D converters for digitizing each of the red, green and blue 
analog video data signals. Three color data signal channels 56a, 56b, 56c connect the 
A/D converter output to the ASIC 26 and 27. A preferred A/D converter is model 9542A 
made by the Raytheon Corporation. The V syn c signal (frame advance signal) output of 
the multimedia source 12 is connected to the ASIC 26 and 27 through a frame advance 
conductor 58. 

As shown in detail in FIG. 7, a pixel value calculator 97 has inputs receiving each 
of the RGB video data signal channels 56a, 56b, 56c. The RGB video data signals are 
connected in parallel to an "OR" gate 99a, and are connected to a 4:1 multiplexer 99b. 
The output of the OR gate 99a is also connected to the multiplexer 99b. The output 101 
of the pixel value calculator 97 is connected through bus 103 to a pixel component 
value comparator in each of the image edge detection 26 and reference frame capture 
27 ASIC circuits. 

Referring now to FIG. 8, wherein a block diagram overview of the functional 

components of the reference frame capture circuit 27 in accordance with one 

embodiment of the present invention is illustrated. As shown, the reference frame 

capture circuit 27 has a microcontroller interface connected to the microcontroller bus 

42, and receives inputs from the line advance signal conductor 53, pixel clock signal 

conductor 54, and the V S y nc frame advance signal conductor 58. The reference frame 

capture circuit 27 also receives pixel data input 101 from the color data signal channels 
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56a, 56b, 56c, through the output 101 of the pixel value calculator 97. Access to a 
current reference frame in stored frame memory is provided through an input port of the 
WRAM 132. A horizontal position counter (HPC) component 105 uses input from the 
line advance conductor 53 and the pixel clock conductor 54 to determine the current 
pi7xel position of the pixel data input 101. The reference frame capture circuit 27 
receives a threshold value input 1 10 used by a pixel value comparator component 115 
to compare the pixel values of the pixel data input 101 to the corresponding pixel values 
of the lines of the current reference frame. The reference frame capture circuit 27 
further includes a reference frame capture switch 120 to trigger the microcontroller 16 to 
capture a new reference frame to store in WRAM 132 stored frame memory for eventual 
transmission to display object 28. 

Operation 

In operation, the analog video signal is digitized in a manner set forth and 
described in U.S. Pat. Nos. 5,767,916 and 5,805,233. Referring now to FIG. 5, the 
display system 20 determines the resolution mode by a firmware program that uses the 
mode identification counter 43. H syn c is input through conductor 45 to the mode 
identification counter 43 and the number of 50 MHz counter clocks over twenty H sync 
pulses is counted. In this way, an average number of clocks per line is obtained. V sy nc 
is input through conductor 47 in the mode identification counter 43 and the number of 
lines for each V syn c pulse is obtained. The firmware then accesses a look-up table that 
determines resolution based on the number of 50 MHz clocks per twenty lines, and 
number of lines per frame. 

Digitization of the analog video data signals occurs based on the n pixel clocks 
per line. Referring now to FIG. 5, the PLL 20, the VCO 50 generates the pixel clock 
signal, and the microcontroller 16 sets the counter 22 to generate a feedback pulse (line 
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advance signal) once every n pixel clocks. Once n is selected, the PLL automatically 
adjusts to produce a line advance signal frequency corresponding to H syn c, and a pixel 
clock signal having a frequency of n times the line advance frequency. 

The PLL works by the phase comparator 46 receiving the H syn c signal from the 
5 delay device 1 8 through conductor 44, and receiving the feedback pulse signal through 
the feedback loop 52. The phase comparator 46 compares the frequencies of the H sy nc 
and the feedback pulse signal, generating an output voltage that is a measure of their 
phase difference. If the feedback pulse frequency does not equal the H sy nc frequency, 
e the phase difference signal causes the VCO pixel clock frequency to deviate so that the 
)S feedback pulse frequency of the counter 22 deviates toward the H syn c frequency. 
J; j Referring again to FIG. 5, the feedback pulse signal (line advance signal) of the 

g counter 22 is directed to the ASIC 26 and 27 through a conductor 53, and the pixel 
W clock signal of the VCO 50 is directed to the ASIC 26 and 27 and the A/D converter 24 
P through conductor 54. The line advance signal and V syn c signal (frame advance signal) 
15 are conditioned to be one clock pulse in duration, through the use of a pulse edge 
O detection circuit or the like. 

The A/D converter 24 samples (reads) the instantaneous voltage value of the 
analog video data signal at the leading edge of each of the pixel clocks, thereby 
generating a series of sampled data signal values. The A/D converter 24 then quantizes 
20 the sampled values by matching each value to one of a series of preselected voltage 
amplitude levels, which have corresponding numerical values. These numerical values 
are then represented digitally and coded to establish 8-bit data for each of the colors 
red, green and blue. 

Referring now to FIGS. 5 and 7, the three eight-bit color data signals are input 
25 through the three respective color data signal channels 56a, 56b, 56c to the pixel value 



M & A FOR NOISE REDUCTION 
USING CAPTURED IMAGES 
EL034440050US DJC/mwb 



- 17- 



) 

Attorney Docket No. 004589. P003 

calculator 97. The output 101 of the pixel value calculator 97 is connected through bus 
103 to the reference frame capture circuit 27. 

Using the output of the reference frame capture circuit 27, the microcontroller 16 
controls the capture of a reference frame from the output of the A/D converter 24. The 
5 captured reference frame is stored in WRAM 1 32 for eventual transmission to the LCD, 
DMD, or other display object 28. At the display object 28, the coded color data signal 
sets pixels to blank (black) or to a specific activated (non-black) status corresponding to 
the sampled voltage level, 
p Referring now to FIG. 8, the output of the reference frame capture circuit 27 is a 

W capture switch 120 that is set to on to instruct the microcontroller 16 to capture a new 
! reference frame, or is set to off to prevent the microcontroller 1 6 from doing so. The 
f|* Vsync frame advance signal 58 generates a first inbound frame that is captured as the 
W first reference frame and stored in stored frame memory in WRAM 1 32. The V sync frame 
P advance signal 58 generates a subsequent inbound frame that is compared by the pixel 
15 value comparator 1 1 5 pixel-by-pixel to the pixel values of the first reference frame. The 
O pixel value comparator 1 1 5 compares the absolute value of the difference between the 
inbound pixel value and the reference pixel value to a threshold value input 110. If the 
absolute value of the difference between any of the corresponding inbound and 
reference pixel values exceeds the threshold value input 1 1 0, then the pixel value 
20 comparator 1 1 5 generates a binary "yes" setting the capture switch 1 20 to on. 

Referring now to FIGS. 7 and 8, in one embodiment the pixel value calculator 97 
determines the value of each pixel by grouping the twenty-four total bits of each pixel 
together in eight groups of individual red, green and blue bits. Each of these groups of 
three bits is passed through the "OR" gate 99a, and the results of all of the eight "ored" 
25 groups and the digital RGB data signals are multiplexed in the 4:1 multiplexer 99b to 

establish a value for each pixel. It is contemplated that in another embodiment, the pixel 
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value may be set in various other ways, including setting a separate value for one or 
each of the RGB digital data signals. In the latter case, the pixel value comparator 115 
compares the absolute value of the difference between each of the RGB values of the 
inbound pixel and the reference pixel to the threshold value input 110. If the difference 
between any of the corresponding inbound and reference pixel RGB values exceeds the 
threshold value input 110, then the pixel value comparator 115 generates a binary "yes" 
setting the capture switch 120 to on. 

Referring again to FIG. 8, the pixel value comparator 1 1 5 performs the pixel 
comparisons iteratively using the HPC 105 to step through every pixel in every line, and 
comparing the absolute value of the difference between each of the inbound pixel 
values and corresponding reference pixel values to the threshold value input 110. If the 
difference between any of the corresponding inbound and reference pixel values 
exceeds the threshold value input 110, then the capture switch 120 is set to on. After 
every pixel of every line of the inbound frame that triggered the capture has been 
completely processed, the capture switch 120 triggers the microcontroller 16 to capture 
a new reference frame. After capturing a new reference frame, the capture switch 120 
is reset to off. 

The entire process of capturing and comparing is repeated continually throughout 
the duration of the image conversion. The result is that the number of reference frames 
captured by the microcontroller 16 for storage in WRAM 132 and eventual display by 
display object 28 is a subset of the total number of inbound frames output by the A/D 
converter 24. Moreover, by selectively capturing an inbound frame only after 
determining that a preceding inbound frame exceeds a certain threshold of change, 
inbound frames whose pixel values are either static or that represent below-threshold 
changes that indicate phase noise or distortion can be eliminated. 
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Example Implementation 

An example implementation of a method for reducing phase noise and distortion 
in a digitally converted image based on reference frame capture will now be described. 
Referring to FIG. 9, wherein a flow diagram of a reference frame capture method used 
in the reference frame capture circuit 27 in accordance with one embodiment of the 
present invention is illustrated. As shown, after advancing the pixel position 150 using 
the HPC 105, the pixel value comparator 115 compares the absolute value of the 
difference between the inbound and reference frame pixel values to a pre-selected 
threshold value 1 10. If the threshold value 1 10 is exceeded, then the capture switch 
120 is set to on 155; otherwise it remains set to off. If the end of the current line of the 
inbound frame 160 hasn't yet been reached, then the pixel position is again advanced 
150, and the pixel value comparison 115 is repeated. However, if the end of the current 
line has been reached, then the next inbound line is first obtained using the line 
advance signal 53. If the end of the inbound frame 165 hasn't yet been reached, then 
the pixel position of the newly obtained next inbound line is advanced 150, and the pixel 
value comparison 115 is repeated, and so on. If, however, the end of the inbound 
frame has been reached 165, then the next inbound frame is obtained using the frame 
advance signal 58. If the capture switch 120 is set on 170, then the newly obtained next 
inbound frame is captured as the new reference frame and stored 175 in stored frame 
memory in WRAM 132. The capture switch 120 is reset to off 180, and another inbound 
frame is obtained using the frame advance signal 58. Since the capture switch 120 is 
now set to off, then the pixel position of the first line in the newly obtained inbound 
frame is advanced 150, and the pixel value comparison 155 is repeated, this time by 
comparing the inbound pixels to the pixels of the newly captured reference frame. 
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Referring now to FIG. 10, wherein a tabulation of a series of inbound frames and 
lines and the corresponding pixel value comparisons and resulting actions is illustrated 
in accordance with one embodiment of the present invention. As shown, each line is 
comprised of pixel values that range in value from hexadecimal zero ("0") to 255 ("FF"), 
with each value corresponding to one of 255 shades of the colors red, green, and blue. 
The position of each pixel with a given line is indicated by the current value of the HPC 
105. The threshold value in this example is pre-set to the value of 5. Lines 1-5 of 
frame 1 is stored as the first reference frame 175. The pixel values of the lines 
comprising inbound frame 2 are compared to the values of reference frame 1 , however 
since none of the differences between the corresponding pixel values of the lines 
comprising frames 1 and 2 exceed the pre-set threshold of 5, no action results. For 
example, the pixels located in reference frame 1, Line 4, HPC 6 and inbound frame 2, 
Line 4, HPC 6 result in the pixel value comparison 115 as follows: 

\2 - 3 | = 1 

Likewise the pixels located in reference frame 1, Line 5, HPC 7 and inbound frame 2, 
Line 5, HPC 7 result in the pixel value comparison 1 15 as follows: 

1 3 - 3 | = o 

Since the difference of 1 and 0 do not exceed the threshold of 5, there is no change 176 
sufficient to trigger the capture of a new reference frame. 

Examining the same relative pixel locations in inbound frame 3232, however, 
yields a different result. This time, the pixels located in reference frame 1, Line 4, HPC 
6 and inbound frame 3232, Line 4, HPC 6 result in the pixel value comparison 1 15 as 
follows: 

| 3 - FF | = FC 

Likewise the pixels located in reference frame 1, Line 5, HPC 7 and inbound frame 

3232, Line 5, HPC 7 result in the pixel value comparison 1 15 as follows: 
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1 3 - FF | = FC 

Since the differences of FC do exceed the threshold of 5, the capture switch 120 is set 
on 155 to trigger a reference frame capture upon advancing to the next inbound frame. 
As shown, advancing to the next inbound frame 3233 results in the action of storing 175 
5 frame 3233 as the new reference frame against which subsequent pixel value 
comparisons will be based. 

As noted earlier, in one embodiment, the individual RGB values are used as the 
pixel values by the pixel value comparator 1 1 5. Referring now to FIG. 1 1 , an example 
of pixel values for an inbound 201 and reference 202 (stored) pixel are shown in 
1® decimal notation. The pixel value comparator 1 15 generates comparisons for the red 
01 color 203, the green color 204, and the blue color 205. As can be seen, only the blue 
p color 205 comparisons result in a value of decimal "1 1 3" that exceeds the pre-set 
y threshold of 5. This is sufficient, however, to trigger the capture and storage of a new 
O reference frame 1 75. 

If Accordingly, a novel method and apparatus is described for reducing phase 

pi noise and distortion in a digitally converted image using a captured reference frame. 

- From the foregoing description, those skilled in the art will recognize that many other 
variations of the present invention are possible. Thus, the present invention is not 
limited by the details described. Instead, the present invention can be practiced with 

20 modifications and alterations within the spirit and scope of the appended claims. 
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I Claim: 

1 1 . A method of displaying a digitally converted image comprising: 

2 receiving a plurality of frames of digitized image data, each frame having a 

3 plurality of pixel values; 

4 capturing at least one reference frame into a frame buffer memory; wherein the 

5 at least one reference frame is selected from the plurality of frames based on the pixel 

6 values of each of the plurality of frames; and 

7 transmitting the at least one captured reference frame to a display object. 

[if 2. The method of claim 1 , wherein the capturing comprises: 

i = i 

§ storing a first one of the plurality of frames as a first reference frame in a frame 

j; :3 buffer memory; 

J4 iteratively comparing a subsequent one of the plurality of frames to the stored 

j| reference frame until the difference between any of the plurality of pixel values of the 

N? subsequent frame and the corresponding plurality of pixel values of the stored reference 

CI frame exceed a pre-selected threshold value; and 

8 storing a next subsequent one of the plurality of frames as a second reference 

9 frame in the frame buffer memory. 

1 3. The method of claim 2, wherein the pixel values comprise a numerical value for 

2 each color of each pixel. 

1 4. The method of claim 3, wherein the difference between any of the plurality of 

2 pixel values is the difference between the numerical values for each color of each of the 

3 corresponding pixels of the compared frames. 
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5. The method of claim 2, wherein the difference between any of the plurality of 
pixel values of the compared frames exceeds a pre-selected threshold value when the 
absolute value of the difference is greater than the pre-selected threshold value. 

6. The method of claim 3 wherein the color for each pixel includes the colors red, 
green, and blue. 

7. An apparatus for displaying a digital image comprising: 

an input for receiving a plurality of frames of digitized image data, each frame 
having a plurality of pixel values; 

a processor for storing a first one of the plurality of frames as a first reference 
frame in a frame buffer memory, for iteratively comparing a subsequent one of the 
plurality of frames to the reference frame until the difference between any of the plurality 
of pixel values of the subsequent frame and the corresponding plurality of pixel values 
of the first reference frame exceed a pre-selected threshold value, and for storing a next 
subsequent one of the plurality of frames as a second reference frame in the frame 
buffer memory; and 

an output for transmitting the stored reference frames of data in frame buffer 
memory to a display object. 

8. The apparatus of claim 7, wherein the pixel values comprise a numerical value 
for each of a plurality of colors of each pixel. 
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1 9. The apparatus of claim 8, wherein the plurality of colors for each pixel includes 

2 red, green, and blue. 

1 10. The apparatus of claim 8, wherein the difference between any of the plurality of 

2 pixel values is the difference between the numerical value for each of the plurality of 

3 colors for each of the corresponding pixels of the compared frames. 

1 11. The apparatus of claim 8, wherein the difference between any of the plurality of 

2 pixel values of the compared frames exceeds a pre-selected threshold value when the 
|;3 absolute value of the difference is greater than the pre-selected threshold value. 

b| 1 2. A digital image display process comprising: 

%2 storing a first frame of data in a reference frame memory, the first frame of data 

" 3 having a plurality of pixel values; 

;<§ comparing a subsequent frame of data to the frame of data in reference frame 

f$ memory until the difference between any of the pixel values of the subsequent frame 

M§ and the corresponding pixel values of the frame of data in reference frame memory 

7 exceed a pre-selected threshold value; 

8 storing a next subsequent frame of data in a reference frame memory; and 

9 transmitting the stored frames of data in reference frame memory to a display object. 

1 1 3. The process of claim 12, wherein the difference between any of the pixel values 

2 is the difference between a numerical value for each color of each of the corresponding 

3 pixels of the compared frames. 
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14. The process of claim 13, wherein the difference between any of the pixel values 
of the compared frames exceeds a pre-selected threshold value when the absolute 
value of the difference is greater than the pre-selected threshold value. 

15. A computer-readable medium having computer-executable instructions for 
performing: 

generating from an analog signal a plurality of frames of digitized image data, 
each frame having a plurality of pixel values; 

storing a first one of the plurality of frames as a reference frame in a frame buffer 
memory; 

iteratively comparing a subsequent one of the plurality of frames to the reference 
frame until the difference between any of the plurality of pixel values of the subsequent 
frame and the corresponding plurality of pixel values of the reference frame exceed a 
pre-selected threshold value; and 

storing a next subsequent one of the plurality of frames as a reference frame in 
the frame buffer memory; and 

transmitting the stored reference frames of data in frame buffer memory to a 
display object. 

16. A method of displaying a digitally converted image comprising: 
generating from an analog signal a plurality of frames of digitized image data, 

each frame having a plurality of pixel values; 

selecting at least one the plurality of frames for display to a display object, 
wherein the at least one frame is selected by: 

storing a first one of the plurality of frames in a reference frame memory; 
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7 iteratively comparing a subsequent one of the plurality of frames to the 

8 reference frame memory until the difference between any of the plurality of pixel 

9 values of the subsequent frame and the corresponding plurality of pixel values of 

1 0 the reference frame memory exceed a pre-selected threshold value; and 

1 1 storing a next subsequent one of the plurality of frames in the reference 

12 frame memory; and 

1 3 transmitting the stored frames of data in the reference frame memory to the 

14 display object. 
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ABSTRACT 

A method for reducing phase noise and distortion in a digitally converted image 
in which only a selected subset of the video frames that represent the image are 
displayed. The selected subset is comprised of reference video frames, where a new 
reference frame is captured for display after determining that a pixel of an inbound video 
frame exceeds a certain threshold of change as compared to a corresponding pixel of a 
previously captured reference frame. The pixel comparison is performed by comparing 
corresponding numerical values representing each of the pixel's color values, red, 
green, and blue, or by comparing corresponding numerical values representing a 
composite color value. The threshold determination is performed by comparing the 
absolute value of the difference between each of the corresponding pixel values to a 
pre-selected change threshold value. After determining that the threshold has been 
exceeded, the next consecutive inbound frame is captured and stored in a frame buffer 
memory as a new reference frame. All of the stored reference frames are transmitted 
from the frame buffer memory for display to a display object in accordance with the 
display object's frame refresh rate. The threshold value is pre-selected to eliminate the 
intervening sub-threshold inbound frames causing the phase noise and distortion, which 
not only improves the quality of the displayed image but also reduces the total number 
of new frames that must be transmitted to the display object. 
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Attorney's Docket No.: 004589.P003 

DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below, next to my name. 

I believe I am the original, first, and sole inventor (if only one name is listed below) or any original, first, and 
joint inventor (if plural names are listed below) of the subject matter which is claimed and for which a patent 
is sought on the invention entitled 

METHOD AND APPARATUS FOR NOISE REDUCTION USING CAPTURED IMAGES 



is attached hereto. 

was filed on as 

United States Application Number 

or PCT International Application Number 

and was amended on 

(if applicable) 

I hereby state that I have reviewed and understand the contents of the above-identified specification, including the 
claim(s), as amended by any amendment referred to above. I do not know and do not believe that the claimed 
invention was ever known or used in the United States of America before my invention thereof, or patented or 
described in any printed publication in any country before my invention thereof or more than one year prior to this 
application, that the same was not in public use or on sale in the United States of America more than one year prior 
to this application, and that the invention has not been patented or made the subject of an inventor's certificate 
issued before the date of this application in any country foreign to the United States of America on an application 
filed by me or my legal representatives or assigns more than twelve months (for a utility patent application) or six 
months (for a design patent application) prior to this application. 

I acknowledge the duty to disclose all information known to me to be material to patentability as defined in Title 
37, Code of Federal Regulations, Section 1.56. 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 119(a)-(d), of any foreign 
application(s) for patent or inventor's certificate listed below and have also identified below any foreign 
application for patent or inventor's certificate having a filing date before that of the application on which priority is 
claimed: 



Prior Foreign Application^ : 



APPLICATION 
NUMBER 


COUNTRY (OR 
INDICATE IF PCT) 


DATE OF FILING 
(day, month, year) 


PRIORITY CLAIMED 
UNDER 37 USC 119 








□ No □ Yes 








□ No □ Yes 








□ No □ Yes 



I hereby claim the benefit under Title 35, United States Code, Section 1 19(e) of any United States 
provisional application(s) listed below: 



APPLICATION 
NUMBER 


FILING DATE 











the specification of which 
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I hereby claim the benefit under Title 35, United States Code, Section 120 of any United States application(s) 
listed below and, insofar as the subject matter of each of the claims of this application is not disclosed in the prior 
United States application in the manner provided by the first paragraph of Title 35, United States Code, Section 
1 12, 1 acknowledge the duty to disclose all information known to me to be material to patentability as defined in 
Title 37, Code of Federal Regulations, Section 1.56 which became available between the filing date of the prior 
application and the national or PCT international filing date of this application: 



APPLICATION 
NUMBER 


FILING DATE 


STATUS (ISSUED, 
PENDING, ABANDONED) 















I hereby appoint the persons listed on Appendix A hereto (which is incorporated by reference and a part of this 
document) as my respective patent attorneys and patent agents, with full power of substitution and revocation, to 
prosecute this application and to transact all business in the Patent and Trademark Office connected herewith. 

Send correspondence to: 

Donna Jo Coningsby, Reg. No. 41,684, BLAKELY, SOKOLOFF, TAYLOR & ZAFMAN, LLP 
(Name of Attorney or Agent) 

12400 Wilshire Boulevard, 7th Floor, Los Angeles, California 90025 and direct telephone calls to: 
Donna Jo Coningsbv. (503) 684-6200. 
(Name of Attorney or Agent) 

I hereby declare that all statements made herein of my own knowledge are true and that all statements made on 
information and belief are believed to be true; and further that these statements were made with the knowledge that 
willful false statements and the like so made are punishable by fine or imprisonment, or both, under Section 1001 
of Title 18 of the United States Code and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 

Alan Lasneski 



Date ftfSd/ac) 

Residence Turner, Oregon USA Citizenship USA 

(City , State) ( Country) 

P.O. Address 4351 Hennies Rd SE 

Turner, Oregon 97392 USA 



Full Name of Sole/First Inventor (given n 



Inventor's Signrfure 




Full Name of Second/Joint Inventor (given name, family name) 



Inventor's Signature Date 



Residence Citizenship 

(City , State ) ( Country) 

P. O. Address 
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A ppendix A 



I hereby appoint BLAKELY, SOKOLOFF, TAYLOR & ZAFMAN LLP, a firm including: William E. 
Alford, Reg. No. 37,764; Farzad E. Amini, Reg. No. 42,261; William Thomas Babbitt, Reg. No. 39,591; Carol F. 
Barry, Reg. No. 41,600; Jordan Michael Becker, Reg. No. 39,602; Lisa N. Benado, Reg. No. 39,995; Bradley J. 
Bereznak, Reg. No. 33,474; Michael A. Bernadicou, Reg. No. 35,934; Roger W. Blakely, Jr., Reg. No. 25,831; R. 
Alan Burnett, Reg. No. 46,149; Gregory D. Caldwell, Reg. No. 39,926; Andrew C. Chen, Reg. No. 43,544; 
Thomas M. Coester, Reg. No. 39,637; Donna Jo Coningsby, Reg. No. 41,684; Florin Corie, Reg. No. 46,244; 
Dennis M. deGuzrnan, Reg. No. 41,702; Stephen M. De Klerk, Reg. No. P46,503; Michael Anthony DeSanctis, 
Reg. No. 39,957; Daniel M. De Vos, Reg. No. 37,813; Sanjeet Dutta, Reg. No. P46,145; Matthew C. Fagan, Reg. 
No. 37,542; TarekN. Fahmi, Reg. No. 41,402; George Fountain, Reg. No. 37,374; James Y. Go, Reg. No. 40,621; 
James A. Henry, Reg. No. 41,064; WiUmore F. Holbrow HI, Reg. No. P41.845; Sheryl Sue Holloway, Reg. No. 
37,850; George W Hoover II, Reg. No. 32,992; Eric S. Hyman, Reg. No. 30,139; William W. Kidd, Reg. No. 
31,772; Sang Hui Kim, Reg. No. 40,450; Walter T. Kim, Reg. No. 42,731; Eric T. King, Reg. No. 44,188; Erica 
W. Kuo, Reg. No. 42,775; George B. Leavell, Reg. No. 45,436; Gordon R. Lindeen HI, Reg. No. 33,192; Jan 
Carol Little, Reg. No. 41,181; Kurt P. Leyendecker, Reg. No. 42,799; Joseph Lutz, Reg. No. 43,765; Michael J. 
Mallie, Reg. No. 36,591; Andre L. Marais, under 37 C.F.R. § 10.9(b); Paul A. Mendonsa, Reg. No. 42,879; Clive 
D. Menezes, Reg. No. 45,493; Chun M. Ng, Reg. No. 36,878; Thien T. Nguyen, Reg. No. 43,835; Thinh V. 
Nguyen, Reg. No. 42,034; Dennis A. Nicholls, Reg. No. 42,036; Daniel E. Ovanezian, Reg. No. 41,236; Kenneth 
B. Paley, Reg. No. 38,989; Marina Portnova, Reg. No. P45,750; William F. Ryann, Reg. 44,313; James H. Salter, 
Reg. No. 35,668; William W. Schaal, Reg. No. 39,018; James C. Scheller, Reg. No. 31,195; Jeffrey Sam Smith, 
Reg. No. 39,377; Maria McCormack Sobrino, Reg. No. 31,639; Stanley W. Sokoloff, Reg. No. 25,128; Judith A. 
Szepesi, Reg. No. 39,393; Vincent P. Tassinari, Reg. No. 42,179; Edwin H. Taylor, Reg. No. 25,129; John F. 
Travis, Reg. No. 43,203; Joseph A. Twarowski, Reg. No. 42,191; Thomas A. Van Zandt, Reg. No. 43,219; Lester 
J. Vincent, Reg. No. 31,460; Glenn E. Von Tersch, Reg. No. 41,364; John Patrick Ward, Reg. No. 40,216; Mark 
L. Watson, Reg. No. P46,322; Thomas C. Webster, Reg. No. P46,154; and Norman Zafman, Reg. No. 26,250; my 
patent attorneys, with offices located at 12400 Wilshire Boulevard, 7th Floor, Los Angeles, California 90025, 
telephone (503) 684-6200, and Justin M. Dillon, Reg. No. 42,486; my patent agent, Dov Rosenfeld, Reg. No. 
38,687; and James R. Thein, Reg. No. 31,710; with full power of substitution and revocation, to prosecute this 
application and to transact all business in the Patent and Trademark Office connected herewith. 
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